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We evaluated the effects of feedback and instruction on resident physician performance during 
developmental surveillance of infants at 2-month preventive care visits. Baseline data were 
obtained by videotaping 3 residents while they performed the physical and developmental exam 
components. Training consisted of individualized feedback and a brief instructional module, 
after which the residents were again videotaped while they performed preventive care visits. All 3 
residents showed improved performance following training. 
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Approximately 17% of all children in the 
United States have a developmental disability 
(Boyle, Decoufle, & Yeargin-Allsoup, 1994). 
To ensure appropriate medical care of these 
children, the American Academy of Pediatrics 
established recommendations for developmental 
surveillance (informal developmental assess¬ 
ment and monitoring) during all preventive 
care visits (Council on Children with Disabil¬ 
ities et al., 2006). The policy outlined five 
components of developmental surveillance, one 
of which is observation of the child through 
physical and developmental examinations. In 
addition, the Accreditation Council for Grad¬ 
uate Medical Education and the American 
Board of Medical Specialties have mandated 
the adoption of competency-based assessment 
by training programs. 

Leake, Barnard, and Christophersen (1978) 
evaluated the effects of training and feedback on 
performance during preventive care visits. 
However, no experimental studies have been 
published on training pediatric residents to 
conduct developmental surveillance. This study 
provides a systematic replication of the proce- 
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dures described by Leake et al. during the 
physical and developmental examination com¬ 
ponent of developmental surveillance at the 2- 
month preventive care visit. 

METHOD 

Participants and Setting 

The study was conducted at an academic 
children’s medical center between August, 
2007, and July, 2008. Participants were 
pediatric resident physicians in their 2nd year 
of training. Recruitment was initiated by 
providing information to the residency class 
regarding the general topic (child development) 
and methodology of the study without disclos¬ 
ing the exact behaviors being observed. Resi¬ 
dents participated voluntarily and signed con¬ 
sent forms prior to enrollment. Seven resident 
physicians, all in their mid-twenties and 
including two men, volunteered to participate; 
however, only three completed the study. (One 
man and three women did not complete the 
study due to personal issues.) Each participant 
was observed and videotaped while performing 
preventive care visits for 2-month-old children 
scheduled through the residents’ general pedi¬ 
atric clinic. Only visits for children with parents 
fluent in English and children without previ¬ 
ously identified genetic syndromes, medical 
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conditions, or developmental delay were eligible 
for observation. Parents provided permission 
for videotaping the visit for research purposes 
prior to the visit. Neither the resident nor the 
family received compensation for participation. 

Dependent Variables 

The videotaped preventive care visits were 
reviewed for the presence or absence of resident 
assessment, either by history or physical exam¬ 
ination, of five specific developmental compo¬ 
nents: (a) the infant’s ability to follow an object 
past midline (the resident was expected to 
attempt to gain the infant’s attention using an 
object or light and attempt to entice the infant 
to follow the object or light past midline); (b) 
the infant’s ability to hold an object placed in 
his or her hand (the resident was expected to 
attempt to place an object such as a tongue 
depressor in the infant’s hand to assess whether 
the infant was able to hold it briefly); (c) the 
infant’s ability to demonstrate a social smile (the 
resident was expected to either attempt to 
engage the infant to elicit a smile or ask the 
parent if the baby smiles at him or her socially); 
(d) the infant’s ability to coo (the resident was 
expected to either attempt to have a “cooing 
conversation” with the infant or ask the parent 
if the infant makes cooing or vowel sounds); 
and (e) the infant’s ability to lift his or her head 
and shoulders off the table while in the prone 
position (the resident was expected to place the 
infant in the prone position and look at the 
infant to assess whether the infant was able to 
complete this skill). The resident’s score was not 
affected by the infant’s ability to complete any of 
these skills, but it did require that the resident’s 
physical examination techniques were used 
appropriately in an attempt to evoke the 
behaviors. For example, the resident was not 
given credit for assessment of the infant’s ability 
to follow an object past midline if the object was 
moved too quickly or held too closely (less than 8 
to 10 inches) to the infant’s face. Components of 
assessment could occur anytime during the visit, 
which was approximately 20 min in duration. 


Design and Procedure 

A nonconcurrent multiple baseline design 
across participants was used to demonstrate 
experimental control. To establish baseline, 
participants were observed and videotaped 
while they performed a minimum of three 2- 
month-old preventive care visits until a stabili¬ 
zation of skills was reached. Stabilization of 
skills was defined as minimal variability in skills 
or absence of a trend toward improvement. 
Once baseline was established, the intervention 
was administered individually to each partici¬ 
pant within 24 hr of baseline. During the 
intervention, an experimenter gave the resident 
individual feedback on his or her performance 
and administered the educational module. 
Individual feedback consisted of outlining the 
purpose of the study and the five targeted 
assessment components. The resident was then 
provided with his or her scores for completed 
visits (e.g., “You completed three of the five 
expected components of assessment. The com¬ 
ponents you missed were the infant’s ability to 
hold an object placed in hand and the infant’s 
ability to coo.”). The educational module was 
lecture based, with video components demon¬ 
strating assessment of milestones expected of 2- 
month-old infants as a model of behaviors. The 
educational module was standardized by using 
PowerPoint slides that contained the videos 
described above and a listing of the develop¬ 
mental milestones so that each resident received 
the same intervention. Both components of the 
intervention were administered during the same 
20- to 25-min period. After exposure to the 
intervention, the resident was observed and 
videotaped while he or she performed the next 
three scheduled 2-month preventive care visits. 
The amount of time that separated each visit 
varied due to scheduling technicalities. 

Data Collection and Reliability 

The videotaped examinations were viewed 
and scored independently by the primary 
investigator and by a subinvestigator who was 
blind to the experimental condition (i.e., 
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baseline or postintervention) of the preventive 
care visit. The reviewers assessed the presence or 
absence of resident surveillance of developmen¬ 
tal status based on the five components outlined 
previously. To obtain a measure of interobserver 
agreement, a second observer reviewed a mini¬ 
mum of 25% of the tapes in each condition 
(baseline, postintervention) for each participant. 
Agreement was calculated by summing the 
agreements on occurrence and then dividing 
by the total number of occurrences measured. 
The two observers disagreed on just two occur¬ 
rences across all participants, both of which 
were the assessment of the infant’s ability to 
follow an object past midline. This resulted in a 
mean interobserver agreement score of 94% 
(range, 80% to 100%). 

RESULTS AND DISCUSSION 

Figure 1 shows the number of assessment 
components completed by each resident during 
the 2-month preventive care visits. None of the 
residents completed all of the components 
during baseline; however, all three residents 
improved their assessment following receipt of 
individual feedback and the educational mod¬ 
ule. The first resident was observed three times 
during baseline and assessed the infant’s ability 
to demonstrate a social smile during all of the 
visits and the ability to coo during two of 
the visits. After the intervention, this resident 
assessed all of the components except the 
infant’s ability to hold an object placed in his 
or her hand during each visit. The second 
resident was observed four times during baseline 
and assessed the infant’s ability to follow an 
object past midline, demonstrate a social smile, 
and coo during three of the visits and the 
infant’s ability to lift his or her head and 
shoulders off the table during two visits. The 
resident did not assess the infant’s ability to 
hold an object placed in his or her hand during 
any of the visits. After the intervention, this 
resident neglected to assess the infant’s ability to 
hold an object placed in his or her hand during 


the first two postintervention visits and did not 
assess the infant’s ability to coo during the third 
visit. The third resident was observed five times 
during baseline and assessed the infant’s ability 
to lift his or her head and shoulders off the table 
during each of the visits. The resident also 
assessed the infant’s ability to demonstrate a 
social smile during four of the visits, coo during 
three of the visits, and follow an object past 
midline during one visit. After the intervention, 
this resident assessed all five criteria during each 
visit. 

The results of this study indicate that 
individual performance-related feedback cou¬ 
pled with a standardized educational module 
that contained video components demonstrat¬ 
ing assessment of milestones is an effective 
method to increase resident physician demon¬ 
stration of competency in developmental sur¬ 
veillance at the 2-month preventive care visit. 
This study is a systematic replication of Leake et 
al. (1978) and successfully extends their work to 
include developmental assessment. A strength of 
this study is that the behavior of the residents 
during the examination was directly measured; 
we did not rely on questionnaires or surveys. 

Residents enrolled in the study on a volunteer 
basis. Therefore, certain characteristics of the 
participants, such as a particular interest in child 
development, future career goals, or a heightened 
motivation to pursue individualized learning, 
may have influenced the results. Similar results 
might not have been found if the study had 
included the entire resident physician class. Also, 
absent follow-up data on resident performance, 
long-term effects of the intervention remain 
unknown. An additional limitation of this study 
was the brief time period over which it was 
conducted; as a result, the baseline lengths were 
staggered by only one visit. 

It should be noted that the residents enrolled 
in this study were not novice pediatric residents. 
All had followed patients and performed 
preventive care visits as part of their weekly 
general pediatric clinic for at least 1 year prior 
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Figure 1. Number of the five components of developmental surveillance of infants conducted by resident physicians 
at the 2-month preventive care visit. 


to enrollment, yet skills displayed at baseline 
were marginal with regard to developmental 
assessment. The education the residents had 
received was in a textbook or other written 
format. No prior formal demonstrations of 
skills or instruction had been provided on this 
topic. Interestingly, the baseline observations 
were quite similar to those reported by Leake et 
al. (1978) 30 years ago. 


Future research should separate the components 
of this intervention to determine if the results were 
due to the performance-related feedback, the 
educational module, individualized attention in 
the area of education, or a combination of the 
three. Future research should also evaluate the 
extent to which maintenance of treatment effects is 
observed by conducting follow-up assessments 
several months after the intervention. 
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